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LOCAL LINEAR QUANTILE ESTIMATION FOR 
NONSTATIONARY TIME SERIES 

By Zhou Zhou and Wei Biao Wu 

University of Chicago 

We consider estimation of quantile curves for a general class of 
nonstationary processes. Consistency and central limit results are 
obtained for local linear quantile estimates under a mild short-range 
dependence condition. Our results are applied to environmental data 
sets. In particular, our results can be used to address the problem of 
whether climate variability has changed, an important problem raised 
by IPCC (Intergovernmental Panel on Climate Change) in 2001. 

1. Introduction. Nonstationary time series analysis has attracted con- 
siderable attention recently; see, for example, Priestley (1988), Dahlhaus 
(1997), Mallat, Papanicolaou and Zhang (1998), Nason, von Sachs and Kroisandt 
(2000), Giurcanu and Spokoiny (2004) and Ombao, von Sachs and Guo (2005), 
among others. Many of the previous results concern estimation of second- 
order characteristics such as covariance functions and time- varying spectra. 

The primary goal of the paper is to estimate quantile curves of a special 
class of nonstationary processes that can be loosely called locally stationary 
processes. Conceptually, local stationarity means that the physical mecha- 
nism generating such processes changes smoothly in time [Mallat, Papan- 
icolaou and Zhang (1998)]. For the purpose of estimating quantile curves 
of locally stationary processes, the classical framework which is based on 
second-order characteristics does not seem to be quite suitable. In partic- 
ular, to estimate quantile functions of heavy-tailed processes, one cannot 
directly apply results which are based on covariance functions. 

Here we shall adopt the following formulation. Let the observed sequence 
{Xi^n}i=i be generated from the model 

(1) Xi,n = G{i/n,Ti), 
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where Tj = (. . . and ej, j G Z are independent and identically dis- 

tributed (i.i.d.) random variables, and G : [0, 1] x M°° i— > M is a measurable 
function such that Ciit) •= G{t, Tj) is a properly defined random variable for 
all t £ [0, 1]. Let F{t,x) =P(Ci(t) < x), a; G M be the cumulative distribution 
function (c.d.f.) of Ci{t), t G [0, 1]; let Qa{t) be the ath quantile function of 
Ci(^), a G (0, 1), namely Qa{t) = i^ix{F{t,x) > a}. We shall estimate Qait), 
the ath quantile curve, from {Xi^n}i'=i by the local linear approach. In the 
sequel for notational convenience we shall write Xj^„ as Xj. 

Process (1) is nonstationary. It covers a wide range of nonstationary pro- 
cesses and it naturally extends many existing stationary time series models 
into the nonstationary setting; see examples in Section 4. Wiener (1958) 
claimed that, for any stationary ergodic process (Xj), there exists a measur- 
able function G and i.i.d. random variables e,, i G Z, such that Xj = G(Tj) 
holds for all i; see also Tong (1990), page 204. The latter relation can be 
interpreted as a physical system with Tj and Xj being the input and the 
output, respectively, and G being the transform or filter that represents the 
underlying physical mechanism. See Wu (2005) for further references. In 
(1), by allowing the data generating mechanism G depending on the time 
index t in such a way that G{t, Tj) changes smoothly with respect to t, one 
has local stationarity in the sense that the subsequence {Xj, . . . ,Xj+j_i} is 
approximately stationary if its length j is sufficiently small compared to n. 

Estimation of quantile curves is an important problem. The report Cli- 
mate Change 2001 written by the Intergovernmental Panel on Climate 
Change (IPCC) addresses the serious problem of whether the climate vari- 
ability or climate extremes have changed. Prominent changes in climate 
variability can have a devastating impact upon the environment. In this 
paper, we shall interpret climate extremes as upper and lower quantiles 
such as (5o.95(i) or (5o.05(i)- By estimating such quantile curves, we obtain 
more detailed distributional information about the underlying process than 
the mean and the variability based on covariances; see Yu, Lu and Stander 
(2003) and Koenker (2005) for more discussions. 

There is a large literature on state domain nonparametric conditional 
quantile estimation where an i.i.d. or stationary series (lj,Xj)"^]^ is observed 
and one is interested in estimating the conditional quantiles of Yi given 
Xj; see, for instance, Bhattacharya and Gangopadhyay (1990), Chaudhuri 
(1991), Chapter 6 of Fan and Gijbels (1996) and Yu and Jones (1998) for 
i.i.d. data and Abberger (1997), Abberger and Heiler (2002), Cai (2002) and 
Cai and Xu (2005) for stationary strong mixing time series. For other contri- 
butions see Cheng and Parzen (1997) and Csorgo and Mielniczuk (1996). On 
the other hand, there are much fewer results on time domain nonparametric 
quantile estimation, and people are mostly interested in the subordinated 
Gaussian process Xj = H{i/n, Zi), i = 1,2, . . . ,n, where (Zj) is a stationary 
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Gaussian process and H isa measurable function; see Ghosh, Beran and Innes 
(1997) and Ghosh and Draghicescu (2002a, 2002b), among others. 

The rest of the paper is structured as follows. Section 2 introduces the lo- 
cal linear quantile estimates, dependence measures and regularity conditions. 
Central limit theorems and Bahadur representations are stated in Section 3 
and proved in Section 6. Section 4 presents examples of nonstationary linear 
processes and nonstationary nonlinear time series models. Applications to 
environmental data are given in Section 5. 

2. Preliminaries. We now introduce some notation. For a vector v = 
{vi,V2, . . . ,Up) e W, let |v| = iJ^i^ivf)^^'^. For apxp matrix A, define \A\ = 
sup{|Av| : |v| = 1}. For a random vector V, write V G (g' > 0) if ||V||g : = 
[E(| V|'^)]^/'^ < oo and ||V|| = ||V||2. Denote by ^ the weak convergence. For 
an interval T C M, denote by i £ N, the collection of functions that 
have ith order continuous derivatives on I, and, for "D C M'^, let CV be 
the collection of real- valued functions that are continuous on P. A function 
/ : M*^ ^ M is said Lipschitz continuous on V cM.'^ if there exists a finite 
constant C, such that \f{xi) — /(x2)| < C\xi — X2I for all xi,X2 G V. For 
a; € M, define x~^ = max(3;, 0). The symbol C denotes a finite generic constant 
which may vary from line to line. 

2.1. Local linear quantile estimator. Observe that Qa{t)^ the ath quan- 
tile function of Cj(i)) satisfies 

(2) Q„(t) = argminE{p„(Ci(t) - /3o) - Pa (0 (*))}, 

/3o 

where pa{x) = ax'^ + (1 — a){—x)^ is the check function [Koenker (2005)]. 
As Qa(ti) ~ Qa{t) + (ii — t)Q'a(t) for ti close to t, it is natural to estimate 
Qa{t) and Q'a{t) via 

n 

(3) ((5Q,fe„(t),Qa,b„(i)) = argmin^/3a(^i - Po - (3i{i/n - t))Kb^{i/n - t), 

{/3o,/9i) i=i 

where K is a kernel function, Ki,^{-) = K{-/bn) and 6„ > is the bandwidth. 
We shall omit the subscript bn in Q and Q' hereafter if no confusion will be 
caused. 

Note that (3) defines a local linear quantile estimate, while the 
analogous version of (2) gives a local constant estimate; see (10). 
Draghicescu, Guillas and Wu (2008) considered local constant quantile esti- 
mation. In the context of local polynomial regression based on mean squared 
errors (MSE), Fan and Gijbels (1996) argued that the local linear estimate 
is generally better than the local constant estimate since the latter suffers 
from the notorious boundary problem. Proposition 1 below asserts that, for 
quantile curve estimation, the local linear approach can also alleviate the 
boundary problem. 
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2.2. Dependence measures and regularity conditions. To study Q in (3), 
it is necessary to introduce appropriate dependence measures. Following Wu 
(2005), we shall adopt physical dependence measures for a family of stochas- 
tic processes. All our results will be expressed in terms of these dependence 
measures. 

Definition 1. Let {H{t,x,Ti)}i^z, {t,x) e [0,1] x M, be a family of 
stochastic processes with H{t,x,Ti) G C^. Let £Q,Sj, j £ be i.i.d. For k > 
0, let T|, = (T_i, Eq, El,. . . , Ek) and define the physical dependence measure 

(4) 5H{k)= sup \\H{t,x,Tk)-H{t,x,Tl)\\. 

(t,x)e[0,l]xR 

Here we recall || • || = []E(| • P)]"*^/^. We say that the system •, Tj)}jgz is 

stable if 

oo 

(5) Y.^H{k) <oo. 

k=0 

Observe that duik) measures the dependence of i7(t,x,Tfc) on the in- 
put £q over {t,x) G [0,1] x M. Condition (5) indicates that the cumulative 
impact of Eq on future values is bounded, thus suggesting short-range de- 
pendence. At a fixed {t,x) G [0, 1] x M, if H{t,x,Ti) is a linear process, (5) 
reduces to the stability condition introduced in Box, Jenkins and Reinsel 
(1994). All the results of this paper will be derived under this short-range 
dependence condition. If the stability condition (5) is violated, namely if the 
sum in equation (5) is infinite, then we have a long-memory process and the 
asymptotic theory developed in the paper does not work. Note that Snik) is 
closely related to the data generating mechanism and hence is easy to work 
with; see examples in Section 4 and also Wu (2005). 

Hereafter we shall assume that K{-) € /C, where /C is the collection of 
density functions K such that K is symmetric with support [—1,1] and K G 
C^[— 1, 1]. A popular choice is the Epanechnikov kernel K{u) = 3max(0, 1 — 
u^)/4. For KgIC let = !\K^{u) du. Define Hj^x = 2 K{u) du if 
J E N is even, and hj^k = Jq K{u) du if j S N is odd. We shall abbreviate 
(j)K (resp. Hj^x) as (p (resp. fij) if no confusion will be caused. 

Note that, for fixed i, {Ci(^)}iez is a stationary process. We shall impose 
the following regularity conditions: 

(Al) For every a G (0,1), Qa{-) £ C^[0,l]- 

(A2) f{t,x) € C([0, 1] X M), where f{t, •) is the density function of Qit). 
(A3) (Stochastic Lipschitz Continuity) There exists C, g > 0, such that 
WUh) - Ci{t2)\\q < C\ti - t2\ holds for all ti,t2 G [0, 1]. 
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Condition (A3) indicates that the underlying data generating mechanism 
G{t,-) changes smoothly in time, thus suggesting local stationarity. Condi- 
tions (Al)-(A3) are needed in the sense that, without them, it might be 
impossible to estimate quantile curves of the series from its stochastic vari- 
ation [see the discussions in Chapter 6 in Fan and Yao (2003)]. We will not 
consider the possibility of jumps in quantile curves in this paper. 



3. Main results. 



3.1. Asymptotic normality. Let ipai^) = ot — I{x < 0} be the left deriva- 
tive of pa{x)- Theorem 1 asserts that, for t G (0,1), nbnf(t,Qa{t))[Qa(t) — 
Qait)] can be approximated by the linear form 

n 

Ta,n{i) := E V'a(^^ - Qa(t) " Q'c.m/n - t))Kt,^{i/n - t) 

i=l 

with an op((n6„)-^/^) error. Due to the linearity it is easier to deal with 
Ta,n{t) —'^\^a,n{t)] which is asymptotically normal under proper conditions. 
Recall (/> = S\ K\u) du and Ci(t) = G{t, T^). 



Theorem 1. A ssumc hyi — > (iTid Tibr^ — > oo . Let t G (0, 1) , J (t, x, T^) — 
I{G{t,Ti)<x} and F{t,x,Ti) =¥{G{t,Ti^i) < x\Ti) . Further assume f{t, 
Qa{i)) > 0, (Al) and (A2), and that the family {F(-, •, Tj)}jg2 is stable. 
Then 

(6) (n6„ + 0(l))/(t, QaimQait) - Qa{t)] " T„,n(t) = 0^{{nbn)'/^) . 

Additionally, assume (A3), nb^ = 0(1) and 



(7) 



a\t):- 



i=0 



>0, 



where Vo{-) =E(-|To) — E(-|T„i). Then the Central Limit Theorem (CLT) 
holds: 



(8) {nbn)^/^[Qo,{t) - Q^{t) - blf,2Q'L{t)/2] ^ N(O,0a2(t)//2(i,Q,(i))). 



Since {F(-, -, Tj)}jgz is stable, a'^{t) exists and is finite; see (37). We call 
a'^{t) the long-run variance which is due to the dependence of the series. 
If J{t,Qa{t),Ti),j € Z, are independent, then cr^(f) = var( J(t, QQ,(f), Tj)) = 
a(l — a). Theorem 1 entails the following corollary on interquartile range 
(IQR), a simple measure for the spread of distributions. 
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Corollary 1. Let IQR{t) = Qo.75{t)-Qo.25{t) and IQR{t) = Qo.75{t)- 
Qo 25(0- -^fit conditions in Theorem 1 he satisfied with a = 0.75 and a = 0.25. 
Let JiQRit,Ti) = J{t,Qo.75it),T,)/f{t,Qo,^5{t)) - J{t,Qo.25{t),T,)/ f{t, 
Qo.25{t)) and assume that ajQR^t) := \\J2i^o'PoJlQRit,'^i)\\ > 0. Then 

(9) {nbny/^[fQR{t) - IQR{t) - blfi2lQR" {t)/2] ^N{0,(l)ajQj^{t)). 

An important issue in applying Theorem 1 is the choice of the bandwidth 
bn- A data-driven procedure is given in Section 3.1.1. 

We now study the boundary behavior of the local linear estimates. In the 
context of quantile estimation for stationary processes, Cai and Xu (2005) 
argued that the local polynomial estimates have a better performance than 
the local constant estimates. Proposition 1 concerns left boundary points. 
Analogous results hold for right boundary points. It shows that the local lin- 
ear quantile estimator alleviates the boundary problem in that the asymp- 
totic MSEs induced by (8) and (12) are of the same order. For local constant 
quantile estimate, however, (11) implies that its rate of convergence is slower 
at the boundary points. Let Qa(0+), (5^(0+), . . . , be right derivatives of Qa 
at 0. 

Proposition 1 (Boundary behavior). Define the local constant estimate 

n 

(10) 0„,fe„(t) =argminVp„(Xi -/3)ii:{,„(i/n-f). 

P i=i 

Under assumptions of Theorem 1 with t = 0, we have: (i) 

(n6„/2)i/2/(0, Qa(0)){Qa(0) - g«(0) - 26„[Q^(0+)/xi + o(l)]} 

(11) 

and (ii) for the local linear estimate Qq(0), 

(n6„)i/2/(0,Q,(0))[Q<,(0) - Q„(0) - blQ'^{Q+)BK /2] 

(12) 

^N(0,a2(0)yK), 

where Bk = (/^i - 4/ii/i3)/(/i2 - 4^i) and V^- = 4 /o^(/X2 - 2fiiu)'^ K{u)'^ du/ 
(/^2-4/if)2. 

3.1.1. Bandwidth selection. For the estimate (5a(-) = (5a,b„(')i b^-^ed on 
(8) of Theorem 1, we define the weighted asymptotic mean integrated squared 
error (AMISE) as 

AMISEiQ^^tJ = f\[bli^2Q'L{t)/2? + [q^a\t)/{nbn)]}f\t,Q^{t))dt. 
Jo 



NONSTATIONARY QUANTILE 



7 



Here the weight f'^{t,Qa{t)) indicates the abundance of data for estimating 
Qa{t)- With more data in the neighborhood of {t,Qa{t)), we can have a 
more accurate estimate and therefore a larger weight is assigned. Observe 
that the optimal AMISE bandwidth 

(13) = ( ^ , ^J>'<"'" I"' 

is of order n~^/^. For independent data, several authors have considered 
the bandwidth selection problem for quantile smoothing; see, for example, 
Fan and Gijbels (1996) and Yu and Jones (1998) (YJ hereafter). Denote by 
6,5f'^(a) the optimal bandwidth obtained under independence. By YJ and 

(13) , 

(14) j#I=ff--'">t'1 



a(l — a) 

Note that a(l — a) = var (J {t,Qa{t),Ti)) and p*{a) is called the variance 
correction factor which accounts for the dependence. YJ proposed a sim- 
ple and easy-to-use rule of thumb selector for b^^'^{a), which we denote by 
bn,Yjict)- Our idea is to use bn{a) by correcting 6n,rj(a) by a factor of 
p*{a), an estimate of p*{a). More precisely, let 

^^"^ (a(l-a))V5' 

where 

^ n-m+l / j+m-1 \ 2 

= r X! i:^ X! ~ ' 



71 — m + 1 \m 



= ipa{Xi - Qa,fe„,yj(a)(i))7 ?n,a = YA=i^i,a/n and 771= [n^/^J . Here is 
a block estimate of long-run variance. See also discussions in Section 3.4. 
Following similar arguments as those of Theorem 5, it can be shown that ct^ 
converges weakly to Jq a'^{t)dt. We suggest 

(15) bl{a) = 6„,yj(a) x p*{a). 

As pointed out by a referee, there are time blocks where the distribution 
of the time series (or a few particular quantiles of interest) changes more 
quickly than others. This suggests using a bandwidth that could change with 
time. In general, variable bandwidth selection can be done by performing 
(15) locally. More precisely, similar to (14), the optimal asymptotic MSE 
local bandwidth bn{a,t) satisfies 



bn{a,t) r a^jt) 
b^'^{a,t) [a{l-a) 



1/5 



p*{a,t), 
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where 6™'^(a,t) is the optimal MSE local bandwidth under independence 
and p*{a,t) is called the local variance correction factor. Here cr^(i) can be 
estimated using (25) in Section 3.4. 

3.2. Bahadur representations. Bahadur representation is an important 
tool for asymptotic analysis of estimators and it provides deep insight into 
properties of estimators by approximating them by linear forms. See, for 
instance, He and Shao (1996), Koenker (2005) and Wu (2007a), and refer- 
ences therein. In this section we shall obtain both locally and globally uni- 
form Bahadur representations for the estimated quantile curve Qa{')- Define 
Oa,nit) = {0a,n,i{t),L,n,2{t)y := (Q„ (i) - Q„ (i) , 6„ (Q^ (t) - Q', (t)))^ . A key 
component to derive Bahadur representation for 6a,n{s) is to study the os- 
cillation behavior of the associated weighted empirical process of form (16). 
Recall that Bahadur (1966) also analyzed oscillations of empirical processes 
and obtained asymptotic expansions of sample quantiles of i.i.d. data. 

Let Zi,„(t) = {l,{i/n-t)/bny . For 6* = (6*1, 6*2)^ G define 



SaAt: ^) =Y.^a(.X, - Q^{t) - a/n - t)Q'^{t) - e^Z,,n{t)) 

(16) 

X Ki,^{i/n-t)zi^n{t). 

Write Sa,n{t) = Sa,n{t,0), where = (0,0)^. Let /^k = diag(l,/i2) be a 2 x 2 
diagonal matrix. We need the following regularity conditions: 

(Bl) Assume that, for k = 0,1,2,3, Fk{t,x,Ti) := d^F{t,x,Ti)/dx^ ex- 
ists and the family {-Ffc(-, •, Tj)}jgz is stable, where FQ{t, x, Tj) = F{t, x, Tj). 

(B2) There exists Co < 00 such that sup(t ,j,)g[o^i]xR -^1(^5 2;, Tj) < Co al- 
most surely. 

(B3) f{t,x) is Lipschitz continuous on [0,1] x M. 

Theorem 2 (Local uniform Bahadur representation). Let < t < 1. As- 
sume that f{t,Qa{t)) > 0, nbn/ log^ n — > 00 and bn — > 0. Then under (Al), 
(B1)-(B3), we have 



sup 

seBn{t) 

(17) 



/ gj logn 2\ 

r-i- + + en]^ 

\ Vnbn J 



nbn 

where Bn{t) = [t- bn,t + bn] n [0, 1] and £>„ = (n5„)~^/2 logn 6^,. 



Theorem 3 (Global uniform Bahadur representation). Let T„ = 1 ■ 
bn]- Assume (Al), (B1)-(B3), infjgp^i] /(t, (5a(t)) > 0, n6,^ ^ 00 and 6„ - 
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0. Then 
(18) sup 



f{t,Qa{t))flKOaAt) 



nbn 



Ov 



1/2 1 
TTn log n 



^nbr, 



+ bnlTn + vr„ , 



where 7r„ = {nbn)~^/'^{logn + {bn)~^^^ + {nbl)^/"^). 



Bahadur representations in Theorems 2 and 3 are useful for studying 
asymptotic properties of the estimated quantile curves by providing asymp- 
totic approximation of 6a,n{t)- The local Bahadur representation of Theorem 
2 is needed in proving consistency and asymptotic normality for a two-step 
smoother given in Section 3.3. By Remark 1 in Section 6, Theorem 3 implies 
global uniform consistency of the estimated quantile curves. We believe that, 
with the uniform Bahadur representation (18), one can construct simultane- 
ous confidence bands for Qa (t) over t£Tn based on a Gaussian approxima- 
tion of {Sa,nit),t £ r„}; see Wu and Zhao (2007) for simultaneous inference 
of mean trends in time series. 

3.3. Two-stage smoothing and jackknife. The estimated curve Qa{t) may 
not be smooth and thus is visually unattractive. To remedy this problem, 
we suggest a second-stage local linear smoothing 

n 

(19) Qa{t) = Y,Qa{i/n)wn{t,i), 

i=l 

where u;„(t,i) = Ki^^{t - i/n)[B2{t) -{t- i/n)Bi{t)]/[B2{t)Bo{t) - Bl{t)] are 
the local linear weights, Bj{t) = Yll=i{'t ~ ~ i/n) and bn is an- 

other bandwidth; see Fan and Gijbels (1996), page 20, for the validation 
of the local linear weights. Fan and Zhang (2000) applied a two-step proce- 
dure to improve the unsmooth raw estimates. Introducing another smooth- 
ing step may possibly bring extra bias and variance. However, fortunately, 
if the bandwidth bn satisfies bn/bn — > 0, then the extra bias and variance 
are negligible compared to those of the original quantile estimator Qa{t). 
The simulation study in Draghicescu, Guillas and Wu (2008) suggests that 
the second-stage smoothing can make the raw estimate visually attractive. 
However the latter paper does not provide theoretical justification of the va- 
lidity of the procedure. Theorem 4 presents properties of the estimator (19). 
The key tool for proving the theorem is the local Bahadur representation 
(17). 

Theorem 4. Assume 0, nbn oo and nb^ = 0(1). Under con- 

ditions of Theorem 2, we have 

(20) ^^6^/(^,Qa(^))[Qa(^) - Qa{t)] - T„,„(t)/y^ = op(l). 
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Hence 

(21) V^lQait) - Qa{t) - blfi2Qa{t)/2] N(0, a\t) c/) K / fit , 

To construct confidence intervals for Qa{t) based on (21), one may need 
to estimate Q'^, which appears highly nontrivial. To circumvent the latter 
problem, we shall propose a bias-corrected estimate. Let K*{u) = 2K{u) — 
2-i/2i^(u/\/2) and 

n 

T^nit) = E - QUt) - Q'.m/n - miii/n - t). 

i=l 

By the argument in (61), we have E(r* „(t)) = o{nb^) since J^u'^K*{u) du = 
0. Let 

(22) Qa,bAt) ■=2Qa,bAt)-Q^,V2bSt)- 

A similar jackknife-type bias corrected estimate is proposed in Wu and Zhao 
(2007) for inference of trends in mean nonstationary models. By (20) and 
following Proposition 6, we have under conditions of Theorem 4 that 

r*„(t) 

Vnbn[Qa,b„{t) - Qa{t)] = y^— + Op(l) 

, , Vnbn 
(23) 

^N(0,a2(t)(/.^,//2(t,Q,(t))). 

With (22) and (23), to construct confidence intervals for Qa{t), we can just 
use Qa,b„(t) and do not need to estimate the bias term 6^^2<5o(i)/2 in (21). 

3.4. Estimation of the density and variance functions. To construct con- 
fidence intervals based on Theorems 1 and 4, we should deal with the non- 
trivial problem of estimating the long-run variance function (T^(t) and the 
density f{t, Qa{t))- By the local stationarity property, we can estimate them 
by using observations Xi for which i/n is close to t. 

For tG(0,l), let s„(i) = max([nt — n6„J , 1), /„(t) = min( [nt -|- n6„J , n) 
and 

(24) Mn{t) = {i(^n:sn{t)<i<ln{t)}. 

Let Zi^a = i^a{Xi — Qa{i/n)). For a sequence m„ with m„ oo and nbn/fnn — 
oo, let 

where Z„(t) = J2ieAr„it) Zi,a/\Ki{t)\ and \Mn{t)\ = /„(t) - Snit) + I is the 
cardinality of Mn{t). Estimator (25) is a localized version of the popular 
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block estimate of the long-run variance. For stationary processes, proper- 
ties of long-run variance estimates have been extensively studied; see, for 
example, Politis, Romano and Wolf (1999). 

For f{t,Qa{t)), we propose the following version of the kernel density 
estimator 

(26) /(t,Q^(t))= / J2 Kl{Qa{t)-X,), 

where K"^ G /C is a kernel and /i„ is the bandwidth satisfying /i„, — > and 
nhn X /i„ — > oo. Theorem 5 asserts consistency of both estimators. Its proof 
is sketched in Section 6. 



Theorem 5. Under conditions of Theorem 2, for any t £ (0,1), (5"^(t) 
and f{t,Qa{t)) are weakly consistent estimators for cr'^{t) and f{t,Qa{t)), 
respectively. 

Choosing optimal smoothing parameters ?n„ and hn is highly nontrivial 
as it involves the issue of nonstationarity as well as dependence. As a rule of 
thumb, we recommend using m„ = 

A*K(t)|V3^ where A* can be obtained 
based on Song (1996), and hn = c^:\J\fn{t)\~^^^ , where c* is chosen by us- 
ing the bandwidth selector of Sheather and Jones (1991). Those smoothing 
parameter selectors perform reasonably well for stationary and short-range 
dependent processes. Since locally (Xj) can be well approximated by a sta- 
tionary process as the window size bn — > 0, it is expected that those selectors 
are also applicable under our setting. 

4. Examples. In this section we present examples of locally station- 
ary linear and nonlinear time series. For such processes, the stability of 
-Ffc(-, •, Tj) as well as the local stationarity conditions can be verified. Hence, 
results in Section 3 are applicable. 



4.1. Nonstationary linear processes. Let be i.i.d. random variables; let 
Oj('))J = 0, 1, . . . , be C^[0, 1] functions such that 

oo 

(27) G{t,Ti) = Y,aj{t)ei^, 

j=0 

is well defined for all t G [0, 1]. Model (27) was considered in Dahlhaus (1997), 
where his primary interest is to estimate time-varying spectra. 

Propositions 2 and 3 are for checking the stability condition and the 
stochastic Lipschitz continuity condition (A3), respectively. They are closely 
related the example in Section 2.1 in Draghicescu, Guillas and Wu (2008). 
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Here for the sake of completeness we provide rigorous and explicit state- 
ments. Let be the distribution function of Si and fs be its density. For 
k = 0,l,. . . , denote by g^*^) (•) the kth derivative of g{-) : M ^ M. Here g^^^ = g. 

Proposition 2. Assume that eq £ q> 0, and its density fs{-) sat- 
isfies 

(28) sup|/W(x)| <Co, A; = 0,1, 2, 3, 

X 

for some Co < oo. Further assume minjg^^i] |ao(OI > 0- q' — min(2,g). 
Then (5Fj^(i) = O(supig[o,i] |ai(t)|^'/^). Hence {Fk{-,-,'^i)]i& is stable if 
Ej°LoSuPtg[o,i]|aj(OI^'/'<oo. 

Proof. We first assume ao(t) = 1. Let G(t,Tj_i) = G(t, Tj) - ej and 
G(i,T*_^) = G(t,T,_i) - Oi(t)eo + ai{t)e'Q. Using min(l,|x|) < \x\i'/'^, we 
have 

5F,{i) = sup sup||FW(n-G(t,T,_i))-Fi'^)(^z-G(t,T*_i))|| 

te[o,i] mgm 

< sup min{2Co,Co||ai(t)eo - ai(t)eo||} 
te[o,i] 

< sup 2C7o|||a,(t)eo-a^(t)e[)|'''/2||=o('sup \a^{t)\'i' 
te[o,i] Vte[o,i] 

The case in which ao(^) ^ 1 can be similarly dealt with by using the argument 
in Section 2.1 in Draghicescu, Guillas and Wu (2008). □ 

Proposition 3. Assume that Eq £ , q > 0. Then condition (A3) holds 
with this q if 

oo 

(29) sup |a;.(t)|™°(2,g) 

Note that Ci{'t)~Ci{s) = Z]j^o[^i(0 ~ o,jis)]£i-j- Using the Lebesgue Dom- 
inant Convergence Theorem and similar arguments as those in the proof of 
Proposition 2, Proposition 3 easily follows. Details are omitted. Note that 
both propositions allow heavy-tailed processes {-'^^j}. 

4.2. Nonstationary nonlinear time series. Let Ei be i.i.d. Many station- 
ary nonlinear time series models are of the form 



(30) 



Zi — R{Zi-i,Ei), 
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where i? is a measurable function [Wu and Shao (2004)] . A natural extension 
of (30) to the locally stationary setting is to incorporate the time index t 
via 

(31) Ci{t)=R{t,Ci-l{t),e^), 0<t<l. 

Theorem 6 asserts that, under suitable conditions, (31) has a unique station- 
ary solution of the form (^j(t) = G{t, Tj). Then one can have a nonstationary 
process Xi^n = G{i/n, Tj). 

Theorem 6. Let q> 0. Assume that, for some xq, sup^g[o,i] ll^(^) ^O) £j)||<j < 
oo, and 



(32) X ■= sup L{t) < 1, where L(t) = sup 



\\R(t,x,eo) - R{t,y,eo)\\q 



tG[o,i] xj^y \x-y\ 

Then for any t G [0, 1], (31) admits a unique stationary solution, and itera- 
tions of (31) lead to Ci{i) = G(t, Tj). Furthermore, we have 

(33) sup ||G(t,T,)-G(t,T*)||, = 0(x*). 
te[o,i] 

Theorem 6 follows from the argument of Theorem 2 in Wu and Shao 
(2004). We omit the proof since there is no essential extra difficulties in- 
volved. By the Markovian structure of {C,i{t)), we can write F{t,x,Ti) = 
F{t, X, C,i{t)). Let -Ffc(t, x, u) = dFk{t, x, u)/du if the latter exists. The follow- 
ing corollary is immediate [cf. Proposition 5 in Wu (2007a)]. 

Corollary 2. Assume that conditions of Theorem 6 hold. Further as- 
sume 

(34) sup sup ]Ffc(t, X, u)] -|- sup sup ]Ffc(t, n)] < oo, /c = 0,l,2,3. 

tG[0,l] ^.-f te[0,l] 

Then we have Spkii) = 0{xi) for some xi G (0, !)• Hence {-?!•(•) •,'^i)}i&z is 
stable. 

Proposition 4. Under conditions of Theorem 6, we have (A3) provided 
that 

sup llM(G(t,To))ll,<oo, 

t6[0,l] 

(35) 



where M{x) = sup 



\\R{t,x,eo) - R{s,x,eQ)\\q 



0<i<s<l \t ~ S\ 
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Proof. We only consider the case q>l. The case q < 1 similarly fol- 
lows. From the proof of Theorem 2 in Wu and Shao (2004), we have 
suPte[o,i] l|G'(t,To)||g < oo. Let Ci = supte[o,i] || Af To))||g and x = 
suptg[o,i]L(t) < 1. For [0,1], by (32), 

\\R{s,Ci-lit),e^) - R{s,Ci-iis),ei)\U < x\\Ci-i{t) - Q-i{s)\U. 

Hence, by (35), \\Q{t)-Ci{s)\U < Ci\t- s\+x\\m -Us)\\,. So (A3) follows. 
□ 

Example 1 (Time varying threshold autoregressive (TVTAR) models). 
Let Ei £ , q > 0, he i.i.d. with distribution function and density fe = F^. 
Consider the model 

(36) Ci{t) = a{t)[Ci^i{t)]+ + b{t)[-Q^i{t)]++e,, 0<t<l, 

where a{t),b{t) G C^[0, 1]. Then Theorem 6 is applicable if sup^gp,!] + 
\b{t)\] < 1. Since F{t,x,u) = Fe{x - a(t)u+ - 6(t)(-n)+), we have (34) if 
there exits a constant C < oo, such that sup„ |/i''"^(u)| < C, A; = 0,...,3. 
Since a{t), b{t) G C^[0, 1], (35) holds. Note that, similarly as the linear process 
case, (Xj) can be heavy-tailed. 

5. Data analysis. Climate change has received enormous attention. The 
understanding of changes in trends, variability and extremes of climate is of 
great importance. See Contribution of Working Group I to the fourth Assess- 
ment Report of the Intergovernmental Panel on Climate Change Solomon 
et al. (2007) (WGI07 hereafter) for a comprehensive discussion. 

The local linear quantile smoothing technique is a useful approach to 
studying changes in trends and variability of nonstationary time series by 
examining quantile curves, for example, the median and IQR curves. As 
pointed out in WGI07, extremes are infrequent events at the high and low 
ends of the range of values of a particular variable. Hence, upper and lower 
quantile curves such as Qo.95(^) a-^d Qomit) are natural tools for illustrating 
changes in climate extremes. 

Robust tools such as quantile methods have already been applied in the 
climatology community. See, for instance, Hannachi (2006). However, previ- 
ous literature is mostly for stationary or integrated linear time series models. 
Additionally, Gaussian or other specific distributional assumptions were typ- 
ically made. These assumptions are vulnerable to model misspecifications, 
which may lead to erroneous conclusions. Here we shall apply our method 
to climate data sets and compare our findings with previous ones. 
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5.1. Global temperature data. Global warming is one of the most im- 
portant issues in climate change and it is closely related to the changes of 
other climate variables (sea level, for instance). Global temperature series 
have been extensively studied in the statistics community; see, for exam- 
ple, Bloomfield and Nychka (1992), Vogelsang (1998), Wu, Woodroofe and 
Mentz (2001) and Wu and Zhao (2007), among others. Most of the above 
studies focused on estimation and inference of trends (mean functions). Less 
statistical research has been done with respect to changes in variability and 
extremes of global temperatures. 

Here we consider the series complied by Jones et al.; see 
http : / / cdiac . esd . ornl . gov/f tp/ trends/temp/ j onescru/. It contains 
global monthly temperature anomalies from 1856 to 2005, relative to the 
1961-1990 mean; see left panel of Figure 2 for the data. We study the median, 
95% and 5% extreme quantile and IQR curves. Jackknife bias correction and 
two-stage smoothing are applied. We tried both the static bandwidth (15) 
and the time- varying 6„(a,t). Both choices of bandwidths yield quite similar 
results. Hence here we present our results by using the static bandwidth. For 
the 0.05th, 0.25th, 0.5th, 0.75th and 0.95th quantile curves, the bandwidths 
are chosen as 0.083, 0.077, 0.075, 0.077 and 0.089, respectively. The band- 
width for the second-stage smoothing step is 0.04. The estimated curves 
and corresponding asymptotic 95% point-wise confidence bands are shown 
in Figure 1. 

From the estimated median curve, we see a clear warming trend. Interest- 
ingly, the increasing of temperature is not homogenous, with a strong faster 
warming trend after 1975. Wu and Zhao (2007) rejected the hypothesis of 
linear increasing of trend. Our estimated median curve is consistent with 
the mean temperature curve in Figure TS.6 in WGI07. Similar trends of 
significant nonhomogenous increase are found in both 5% and 95% quan- 
tile curves of monthly temperature (although 95% quantile curve seems to 
slightly decrease after 1995). As summarized in WGI07, 

"Changes in extremes of temperature are consistent with warming. Observa- 
tions show widespread reductions in the number of frost days in mid-latitude 
regions, increases in the number of warm extremes (warmest 10% of days or 
nights) and a reduction in the number of daily cold extremes (coldest 10% of 
days or nights)." 

Note that the magnitude of increase of the 5% quantile is approximately 
twice that of the 95% quantile, indicating reduction in the temperature 
variability. This result is consistent with the statement in Climate Change 
2001: The Scientific Basis [IPCC (2001), WGIOl hereafter], that while in- 
creases in the frequency of warm days have been observed, decreases in the 
number of cool nights have been stronger. Reduction in the variability can 
be verified by the IQR curve. Compared with the width of the confidence 
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band, the reduction of variability is not as significant as the increasing of 
median and extreme quantiles. In Chapter 2 of WGIOl several climatological 
papers were cited to confirm that annual and monthly variation of global 
temperature had decreased. To summarize, although based on different sta- 
tistical models and assumptions, results of our method are consistent with 
the previous empirical findings. Our results suggest that global monthly 
temperature distribution in the last 150 years has the tendency of shifting 
to the right with shrinking variability. 

5.2. U.S. precipitation data. This data set contains spatially averaged 
monthly total precipitation in the land surface of United States from January 
1895 to April 2007. It is available at National Climate Data Center's website 
at http://wwwl. ncdc.noaa.gov/pub/data/cirs/drd964x.pcpst.txt. One can 
refer to http : //wwwl .ncdc.noaa.gov/pub/data/cirs/ state .README for a 
detailed description. 

Average seasonal trend was first removed from the data and we focus 
on inference of the anomalies; the right panel of Figure 2 summarizes the 
anomalies. Jackknife bias correction and two-stage smoothing are applied. 
We also choose the static bandwidth (15) since the time- varying bandwidths 
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Fig. 2. Time series plots of the data examples. Left panel: global warming data. Right 
panel: U.S. precipitation data. 

provide similar estimated curves. For the 0.05th, 0.25th, 0.5th, 0.75th and 
0.95th quantile curves, the bandwidths are chosen as 0.114, 0.096, 0.096, 
0.096 and 0.115, respectively. The bandwidth for the second-stage smoothing 
step is chosen as 0.05. The estimated median, extreme quantile and IQR 
curves and corresponding asymptotic 95% point-wise confidence bands are 
shown in Figure 3. 

In theory, increased temperature is likely to produce heavier precipitation. 
As stated in WGIOl, 

"Increasing global surface temperatures are very likely to lead to changes 
in precipitation and atmospheric moisture, because of changes in atmospheric 
circulation, a more active hydrological cycle, and increases in the water holding 
capacity throughout the atmosphere." 

However, the estimated median curve shows that even though there is a 
slight increase of the monthly precipitation, noticeably after 1960, the trend 
is not significant with respect to its stochastic uncertainty. WGIOl sug- 
gested to compare changes in many of the moisture-related variables, such 
as streamflow and soil moisture, with precipitation to help validate long- 
term precipitation trends and claimed that the precipitation change was not 
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Fig. 3. Trends in median, extreme quantiles and IQR of monthly U.S. precipitation, with 
95% point-wise confidence bands. 



spatially uniform. On the other hand, one factor which could be consid- 
ered is the so-called "global dimming" effect [see, e.g., Stanhill and Cohen 
(2001), Roderick and Farquhar (2002), Liepert et al. (2004), among others], 
which reduces the sun radiation and water evaporation on the surface and 
therefore reduces precipitation. Hence future research taking into account of 
those factors may help confirm (or reject) the heavier precipitation hypoth- 
esis. Both WGI07 and WGIOl claimed that there was an amplified increase 
of heavy precipitation, even in regions where the mean precipitation de- 
creased. Our estimated 95% quantile curve does show a stronger increase 
than the mean, but still not very impressive considering the width of the 
confidence band. The 5% quantile and IQR curves also show no significant 
increase or decrease pattern. To summarize, the U.S. monthly precipitation 
data does not show significant increasing trend of precipitation median and 
heavy precipitation, as reported in the literature and a more comprehensive 
study including more factors may be helpful. 

6. Proof of results in Section 3. Unless otherwise specified, we will only 
prove results in Section 3 for a = 1/2, since results for other quantiles simi- 
larly follow. We shall also omit the subscript a in the notation if no confusion 
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will be caused. Here we list some notation that we will use in this sec- 
tion. Let Yi{t) = Xi - Q{t) - {i/n - t)Q'{t), Yi{t) = G{t,Ti) - Q{t), hit) = 
'>p{Yi{t)) and Sn{t) = Td=i^ii't)^iA''^)^ where Vj,„(t) = Kb^{i/n - t)zi,„(t). 
Recah Zj_„(t) = (1, (i/n - t)/6n)'^- 

For k £ Z, define the projection operator Vk on C^, by Vk- = E(-|Tfc) — 
E(-|Tfc_i). Recah J{t,x,Ti) = I{G{t,Ti) < x}. Let i > 1. Then E{J{t,x, 
Ti)|To) =E(F(i,x,T,_i)|To) andE(J(t,x,T,)|T_i) =E(F(t,x,T,_i)|T_i). 
Since eQ,ej,j E Z, are independent, we have E{F{t, x, Tj_i)|T_i) = K{F{t, x, 
T*_i)|T„i) and E(F(t, x, T*_i)|T„i) = x, T*_i)|To). Hence 

||PoJ(t,x,T,)|| = ||E(F(t,x,T,_i)-F(t,x,T*„i)|To)|| 

(37) 

< \\F{t,x,Ti_i)-F{t,x,TU)\\- 
Since {F(-, •, Tj)}jg2 is stable, by (5), a{t) in (7) is finite. 

Lemma 1. Let bn — > 0; let (ani)F=i triangular array of real numbers 
such that 

n 

(38) Ar,{t) ■.= Y,alKlii/n -t)<C, 

i=l 

and Wn '■= maxjg^^(i) |a„j| — > 0. Assume (Al), (A2) and that {F{-, •, TJjjg^ 
is stable. Let r]ni{t) = [p{Yi{t) - ani) - p{Yi{t)) + aniil^{Yi{t))]K}j^{i/n - t). 
T/ien varEr=i??m(i)]^0. 

Proof. Let A = [inf^gjo,!] Q{t) - l,supjg[o^i] Q{t) + 1]. For every /c > 1, 
sufficiently small eo and sufficiently large n, we have by (37) that, for |e| < eo, 

SUP||P,„fc[^(yi(t) - e) - i^{Y,{t))\\\ < SVF2\\Vi.kmi{t) " 

(39) 

<26F{k-l), 

where SUP denotes sup|^|<^g sup,jg_^^(j-). On the other hand, for sufficiently 
small eq and sufficiently large n, by condition (A2), 

S\JP\\Vi-kmYi{t) - e) - HY,m\\ 
(40) <sup||^(y,(t)-6)-V'(y,(t))|| 

< en max fit.x) <CeJ . 

Let Zk,n{s) = YA=i'Pi-k[^{Yi{t) - soni) - 'ip{Yi{t))]aniKb^Ai / - t)- Since 
the summands of Z^^nis) are martingale differences, by (38), (39) and (40), 
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we have, for large n, 

sup \\Zk,nis)f= sup WVi-kMyiit) - sani) - ^l^iY,{t))]f 

(41) xaliKlii/n-t) 
< C mm{(5^(A: — 1), 

holds for > 1. Clearly, supo<j,<i ||.^o,n('5)|P < C mm(l, Since {F{-, •, Tj)} 
is stable, Yl'k=i^F{k — 1) < oo. By the Lebesgue Dominated Convergence 



Theorem, we have 



Y}nni{t)-^11m{t)] 
i=l 



CO l.\ CO 
^ / Zk,n{s)ds <J2 / 



Zk,nis)\\ ds 



(42) 



< Cmin(l,ro„) + ^Cmin{5F(A;- l),roy^} 

A;=l 

^0 



as n — > oo in view of ro„ — > 0. □ 

Lemma 2. Assume (Al), (A2) and bn 0. Then for all large n, 
sup \E[p{Y,{t) + a) - piY^it)) - ail;{Yim - f{t, Q{t))a^/2\ 

= o{a^) as a ^ 0. 



(43) 



Proof. Let T G N and r = a/T. Since p{x) is convex, for all j = 1, ... , T, 
we have 

r[^(y,(t)+iT-r)-^(y,(t))] 
(44) < p{Yi{t) +jT) - p{Y,{t)+jT - r) - i^{Y,{t))r 

<T[i,{Y,{t)+jT)-ij{Yi{t))]. 

Taking expectation, using (A2), summing over j = 1, . . . , T, and then letting 
T — > oo, we have (43). □ 

Before we proceed to the next proposition, let us first observe that the 
original estimation problem (3) is equivalent to the following transformed 
one: 

(45) 



avgrnm^ p{Yi{t) - 'd'^ Ui^n{t))Kb^{i/n - t) 



i=l 
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where 

(46) U,,„(t) := {ui,n,l{t),Ui,n,2{t)V = ^n{t)-^^\l, {i/n - t)/bny, 

and ^nit) = Y:i=ii'^,ii/n - t)/bn)^ il,{i/n - t)/bn)Ki,^ii/n - t). Since KG 
/C, we have = nbnfiK + 0{1) uniformly on any closed interval of (0, 1). 

The estimators {Q{t),Q'{t))~^ of (3) and are related via 

Mt) = ^n{ty^\Q{t)-Q{t)) and 

(47) 

Mt) = ^nit)^^^bn{Q'{t)-Q'it)). 
Recall e{t) = {Q{t) - Q{t),bn{Q'{t) - Q'(i)))^ and (16) for Sn{t). 

Proposition 5. Assume (Al), (A2), bn^O, n6„, ^ oo, f{t,Q{t))>0 
and that {F{-,-,Ti)}i^z is stable. Then 

(48) fit, Q(t))S„(t)^(t) - Sn{t) = op((n6„) 1/2). 

Proof. For a fixed vector 'd G M^, let Oni = '!?"'' Uj_„,(t). Then 
maxjg_^^(() |a„i| 0. Note that X^Li Ui,n(t)u^„(i)i4:b„(z/n - 1) = 1^2, where 
Id2 is the 2x2 identity matrix. Let zui{t) = p{Yi{t) — {)'^Ui^n{t)) — p{Yi{t)) + 
{f^ VLi^n{t)'^iXi{t)) ■ By Lemma 2, we have 

(49) x(l + o(l))Kt.{i/n-() 

Since K £ IC,hy simple calculations, (38) holds. Thus by Lemma 1 and (49), 
we have 

(50) ^^,(t)K,Ji/n-t)^/MM!?l! in probability. 
1=1 ^ 

Note that both sides of (50) are convex functions of '&. By the Convexity 
Lemma in Pollard (1991), page 187, we have that the above convergence 
holds uniformly for \d\ <c, where c > is a finite constant. Now the relation 
(48) easily follows from properties of convex functions; see, for example, the 
proofs of Theorem 1 in Wu (2007a), Theorems 2.2 and 2.4 in Bai, Rao and 
Wu (1992) or Theorem 1 in Pollard (1991). Details are omitted. □ 

Lemma 3. Let Vnit) = cov(S'.„(i)) and z/k = diag(/^j^ K'^{u) du, j\ v? x 
K'^(u)du). Suppose that {F(-, •, Tj)}jgz is stable, 5^ — > and n5n — > oo. 
Then 

(51) inbn)-^Vnit) ^ a\t)i^. 
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Proof. For simplicity we write Cj etc for ej(t). Clearly (ej)jgz is sta- 
tionary and E(ei) = 0. Let A = E^i^iCj and Ri = ei- Di. Recall (24) for 

Mn{t). Let Sn = Snit), In = ln{t), Ai = 'Ej^l{ej-Dj) and Bn = EieAr„(t) Avi, 
By the summation by parts formula, 



\Snit) - B„ 



In S n 1 



1=0 



Let = T!k=m^F{k - 1). By (3) of Theorem 1 in Wu (2007b), we have 



max ||j4JP<C = o(L — s„) = o(n6n 

0<i<«„-s„ ^ 



since 0^ 



as m ^ oo. Hence 



(52)||5„(t) -BJ< UjI,/^ \^i,n - Vi+l,„| + |V;„_,„,„| = o{{nbn)^^^) 



i=0 



Note that Di are martingale differences. By orthogonality, since K £ IC, it is 
easily seen that {nbn)~^^{BnBj) cr^(t)fK- Hence, by (52), we have (51). 
□ 



Lemma 4. Under conditions of Lemma 3 and assuming a{t) > 0, we 
have 



(53) 



{nbn)-^/^Sn{t)^'N{0,a\t)uK). 



Proof. For m G N, let em,i = E(ei|Ti_m,i), where, for j < k, Tj^k = 
(ej, Gj+i, . . . , Efc). Here we also write etc for ei{t). Then {em,i)i'=i is rn- 
dependent. Elementary calculations show that Vi-kem,i = E(Pj_fce^|Tj_m,^)■ 
Thus, we have for sufficiently large n, 

sup ||7^i-fc(ei - em,i)|| < 2 sup \\Vi^kei\\ <26F{k - 1). 



On the other hand, by the stationarity of (ej), 

sup \\Vi-k{ei - em,i)\\ ^ sup \\ei-em,i\\ = 

Clearly limm-^oo Tj{m) = 0. By the orthogonality, 



|eo - em,o\\ ■= rj{m). 



E 



(54) 
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< Cn6„min{rj(m), (5|'(/c — 1)}. 



Write Sm,n{t) = T.':=iem,i^i,n- Then5„(0-5™,„(t) = Er=oE?=i^i-fe(e*- 
em,i)^i,n- Let LIM denote linim^oo limsup^^o^. Since •, Tj)}jgz is sta- 
ble, as (42), by the Lebesgue Dominated Convergence Theorem, 



LIM 



LIM- 



(nfe„)V2 



(55) 



1=1 



< LIM(n6n)-^/^ 

k=0 

Cmin(l, rj(m)) + ^ C min{ Tj(m), 2Jf (A; - 1)} 



<LIM 



0. 



A;=l 



For a symmetric matrix A let A(^) be its smallest eigenvalue. Let Vm.nit) = 
cov {Sm.^n{t))- By (55) and Lemma 3, LIM|cr^(t)fK — in'bn)~^Vm,n{'t)\ = 0. 
Note that (T^(t)i/K is nonsingular. Thus there exists an tuq £ N and cq > 
such that for m > tuq and for all large n, {nbn)~^ X{Vm,n{'t)) > cq. Note that 
i^m,i) are m-dependent. Applying the Cramer-Wold device and the central 
limit theorem for m-dependent random variables [see Hoeffding and Robbins 

(1948)], we have V;;^,i/^(t)[5^,„(t) - E5™,„(t)] N(0,Id2) for fixed m > mo- 
Thus by (55), we obtain (53). □ 

Proposition 6. Under conditions of Theorem 1, we have 

(56) {nbn)-'^^[Sn{t)-ESnit)]^N{0,aHt)uK). 

Proof. First note that 

(57) sup \\Vi-k{ei - ei)\\ < sup (||Pi-fcei|| + ||Pi_fcei||) < 2(5f(A; - 1). 

Let n be sufficiently large and e > be sufficiently small. Then for i G A/'„(t), 

\\Vi-k{ei - ei)\\ 

— II ^2 II 

<\\{e,-e,)I{Ui/n)-C{t)\>e}\\ 
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+ \\{e,-e,)I{\Q{iln)-m\<^]\\ 

< WnUi/n) - Q{t)\ > 6}|| + \\I{\Ci{t) - Q{t)\ < Q'{t)hn + e}\\ 

< 6-«/2||0(z/n) - m\f + {2Q'm, + 2ef^nt, e), 

where e) = sup{/(t, x) : x G (Q(i) - Q'(t)6„ - e, Q(t) + + e)}. Let 

e = hn ■ By (A2), (A3) and the above inequahty, we have 

(58) ||P,_fc(e, - e,)|| < C[\i/n - t\^l^ jh^^ + {hTf''] < C6--(V3,9/6) 
for ah i £Mn{t)- Hence by (57) and (58), we obtain 

m^k{e^ - e.)ll < mm{26F{k - l),CbT''^'/''''^''^}. 
Note that 6„ — > 0. Using the same argument for (42), we have 

(59) \\Sn{t) - ESnit) - Sn{t)\\ = o{{nhnY'^). 

Now by Lemma 4, (56) holds. □ 

Proof of Theorem 1. Clearly Proposition 5 implies (6). By Proposi- 
tion 5, we have 

(60) fit, Q{t))fiK{nbn) ' 9{t) Jnby72 (^^b )V2 " 

By (Al), (A2) and Taylor's expansions, since a = F{i/n,Q{i/n)), we have 

n 

^Snit) = Qii/n)) - Fii/n, Q{t) + Q'{t){i/n - t))]vi,„(t) 

(61) 

= nblf{t, Q{t))Q"{t){l + o(l))(^2, 0)^/2 + o{nbl). 
By (60), (61) and Proposition 6, we have (8) in view of nb^ = 0(1) and 

(62) (n6„)i/2/(i,Q(i))[^K^>) - 6^Q"(t)(/i2, 0)^/2] ^N(0,a2(t)z.K). □ 
Proof of Corollary 1. By (6), we have 

V^n[Q.{t) - Qcit)] - ^=%^^L^[1 + 0((n6„)-i)] = op(l). 

Applying the above relation with a = 0.75 and a = 0.25, we have (9) by 
using the arguments in the proofs of Theorem 1 and Proposition 6. □ 

Proof of Proposition 1. We shaU first prove (h). A careful check of 
the proofs of Lemmas 1, 2 and Proposition 5 implies that they are also valid 
for t = 0, and (48) becomes 

(63) /(O, Q(0))S„(0)^(0) - 5„(0) = o^{{nbnY'''). 
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Also Lemmas 3, 4 and Proposition 6 are still valid with z/k therein replaced 
by the 2x2 matrix = {^\j-j'\)i<j,j'<2^ where ipj = Jq K'^{u) du. By 
(Al), (A2) and Taylor's expansions, since K £ IC, with elementary calcula- 
tions, (61) becomes 

[nb„\ 

ESn{0)= J2 [F{i/n,Q{i/n)) - F{i/n,Q{0) + Q'{0)i/n)]^r^,n{0) 
1=1 



J2[Q{i/n) - (Q(0) + Q'{0+)i/n)]f{i/n, Q{i/n)) 



(64) 



1=1 



X (l + o(l))Jf6jz/n)vi,„(0) 
"/(0,Q(0))Q"(0+)(l + o(l)) 



2 







»i 

y y U'^K{u) du j 

Additionally, since K £ KL, T,n{0) / (nbn) — >■ F, where F is a 2 x 2 matrix 
with its (j,j')th entry being /g u^^~^'^K(u) du, 1 <j,j' < 2. Hence, by (63), 
/(0,Q(0))^(0) - S-i(0)5„(0) = op((n6„)-i/2). As in the proof of Theorem 
1, the latter relation implies (12) by Slutsky's theorem and (64), since 
(n6„)-V2[s„(0) _E5„(0)] ^N(0,c72(0)4). 

We now prove (i). Let ifn ■= J27=i ^'^bn{i/n) = {nbn)/2 + 0(1) and A„ = 
^27=1 i^i^i ~ Qi^))^'^b„ i'i'/n')- Using the argument in the proof of Proposition 
5, we have the following analogous version of (63): 

(65) /(O, Q(0))v^„(Q(0) - Q(0)) - A„ = op{{nb„)^/^). 

Similarly as Proposition 6, the CLT [A„ - E(A„)]/Vra6^ ^ N(0, cr2(0)y?o) 
holds. As (64), 

[nbn\ 

E(A„)= [Fii/n,Q{i/n))-Fii/n,QiO))]K,„ii/n) 

i=l 
n 

= T^iQi^M - Qmfi^/n, Q{^/n))il + o(l))/^6„ (i/n) 

i=l 

= n52/(0,Q(0))Q'(0+)(l + o(l))/ii. 
Hence (11) holds by elementary manipulations. □ 

We will only prove Theorem 3 in this section since Theorem 2 can be 
proved by using the same technique. Let 

n 

Unit, 6) = Y^mm) - e'^Zi.nit)) 
1=1 
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(66) 

- E[V^(y,(t) - eTz,,„(t))|Ti_i]}vi(t), 
where Vi(t) = {vi^i{t),Vi^2{t)V = Kb„{i/n - {i/n - t)/bny, and 

n 

Nnit, e) = ^{E[^(y,(t) - ^Tz,,„(t))|Ti_i] 

(67) 

-E^(y,(t)-0Tzi,„(t))}vi(t). 

Then 

5„(t,e) -E5„(t,e) = M„(t,e) + iV„,(t,0). 

Note that the summands of Mn{t,9) form a martingale difference sequence 
while summands of Nn{t,9) are differentiable with respect to t and 9; see 
Lemmas 5 and 6 below. Decomposing Sn{t,9) — ES„(t, 0) into a martingale 
part and a differentiable part is useful for studying its oscillatory behavior. 
The technique in Wu (2007a) is useful. 

Proof of Theorem 3. Let Sn{t,9) = ESn{t,9) and Sn{t) = E5„(t). 
For 7n — > 0, we have 

sup \Sn{t,9) -Sn{t,9)- [Sn{t) - 5„(t)]| 

i6r„,|0|<7„ 

= Op((n6„y/2(^i/2 iog„ + ^^5-1/2) + „-3) 
in view of Lemmas 5 and 6. By Lemma 7 below, we have 

sup \Sn{t,m)-Sn{t,m) - [Sn{t)-Sn{t)]\ 

(68) 

= Op((n6„) ^2(^1/2 iog„ + 7r„&-i/2)). 
Elementary calculations using (Bl), (B3) and Lemma 7 show that 

(69) sup \Sn{tAt))-Snit)^nhn!{t,Q{t))iiySit)\ = Op(n62 7r„ + nfe^vr^ ) . 

Since {nhnY^'^T^nbn^^'^ = 0(n6„7r^), by (68), (69) and Lemma 8, Theorem 3 
follows. □ 

Lemma 5. Let (7n)neN positive with 7„ — > 0. Under conditions of 
Theorem 3, we have 

(70) sup \Mn{t,e)-Mn{tM=0^{{nhnlnf'^logn + n-'^). 

te[0,l],|6i|<7„ 
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Proof. Let Mn{t-) = (0,nt] nA/'„(t) and A/'„(t+) = {nt,n]f^Mn{t). De- 
fine 

e) = ^{Yi{t) - 9'^zi,n{t)) - E[ij{Yi{t) - e^Zi,„(t))|T,_i]. 

Write Mn,i{t, 9) = j:^^rii{t, e)vi,i{t), Mn^t,^) = EieMAt+)'l^(t^^>^^^^^ 
Mn,3{t,0) = EieX^(t-)ViAt>iM't)- Since M„(t,0) = (M„,i(t,e), 

Ej=2 0)V , it suffices to prove (70) with therein replaced by M^j 
for j = 1, 2, 3. For presentation clarity we consider j = 2. The other two cases 
can be similarly dealt with. 

For a real sequence {gn)n£N with (?„ > 3 for all n, define Un = 

{nbn7n)^^'^gn/loggn, (pn = {nhnJuY^'^gr^Ogn, ai{t,e) = 4^{Yi{t)-e~''zi^n{t)) X 

Vi^t), and 

An(t) = max sup \ai(t,9) — ai(t,0)\, 
ieAf„{t+) \e\<j„ 

Un{t)= nVl'{y^{t) + \^i,n{t)\ln) 



By the monotonicity of 

(71) sup E[(r?i(i,^)-r?,(t,0))V,2|Ti-i]<C/„(0- 

By (B2), (B3) and since 7^ — > 0, it is easy to see that, for sufficiently large 
n, 



E 


sup Un{t) 


< 3n6n7„coE 


sup Fi(t,x,Ti_i) 


< Cnbn'Jn, 




Ue[o,i] J 




-(t,x)e[0,l]xR 





where cq = j\ K'^{u)w^{l + u^)^/^ (iu < 00. Hence by Markov's Inequality, 

0(5'n^log5'„) = 0(1). 
Similarly, using An{t) < sup\g\<^^^J2ieAfnit+) Wi(t^^) -ai{t,0)\, we have 



(72) P 


sup Un{t) > ul 




sup Unit) 




Ue[o,i] J 




Ug[o,i] -1 



(73) 



sup An{t) > Un 
.tG[0,l] 



:0(1). 



Let l = n^, Qi = {iki/l,k2/l):\h\,\k2\ < n^;ki £ Z,i = 1,2} D {[-^n,ln] x 
[— TniTn]} and Hi = {ks/l-.O < ks < n^;ks G Z}. Then Qi x TLi has at most 



27n^^ points. 
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By the argument in the proof of Lemma 4 in Wu (2007a), we obtain in 
view of Freedman's (1975) exponential inequahty for martingale differences 
that 

P( sup \Mn,2{t,e)-Mn,2{t,0)\>2^n, 

(74) sup An{t) < Un, sup Un{t) < I 

iG[0,l] te[o,i] J 

< 27n27o[exp{-02/(4tx„0„ + 2ul)}]. 
Since 4ti„</)„ log?i = o((/>^) and 2ti^ logn = o((/)^), (72), (73) and (74) imply 

(75) lim p( sup \Mn,2it,e) - M„,2(t,0)| > 2<Pn\=0. 

Next we shall apply a chaining argument. For x = {xi,X2)'^ S K^, define 
[xji := (La;iJ;, [x2\iV and [x]; := {\xi]i, [x2l/)'^, where [u\i = [ul\/l and 
\u]i = Let 6 £ [— 7n,7n] x [— 7n,7n]- Since ip{-) is nondecreasing, we 

have 

(76) ai(t,\e]i) <ai{t,9) <ai{t,[e\i) for alH G and i G A/'„(t+). 

Let Lj = sup(t^,)g[o i]xR-^i(i,a;,Tj) and K = Er=i Let |si|,|s2| <7n- 
By (B2), for all large n and i G Afn{t+), \E[{i^{Yi{t) - si) -i;{Y,{t) - S2))\Ti_i]\ < 
— S2|. Since \6 — \9~\i\ = 0{l~^), we have 

(77) sup mMt,e)-ai{t,\e-]i)]\Ti,^i}\<ci-^Vn. 

The same inequality holds if we replace \6~\i by \_6\i. Therefore, for all \9\ < 

7n, 

M„,2(i, \9-]i) - Mn,2{t,0) - Cl-^Vn < Mn,2{t,e) - M„,2(i,0) 

< Mn,2{t, [e\i) - M„,2(t,o) + crV„. 

Since ^iVn) < Cn, l~^Vn = Op(n~^). Since gn ^ oo can be arbitrarily slow, 
we have 

(78) sup \Mn,2it,9)-Mn,2{tM = Owi{nbnjn)^/Hogn + n-'^). 

teHi,\e\<'y„ 

Let tk = k/l, k = 0, . . . ,1. By the triangle inequality, we have 

sup \Mn,2{t,e)-Mn,2{t,0)\ 

t6[o,i],|e|<7n 

(79) < max sup \Mn,2{t,9) - Mn,2itk,9)\ 

O<fc<«-lo<t„tj^<i-i,|0|<^„ 
+ sup \Mn,2{t,9)-Mn,2{t,0)\. 

teHi,|0|<7n 
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By the similar chaining argument as those in the proof of (78) as weh as the 
fact that Fi{t,x,Ti) is bounded on [0, 1] x M, we have 

(80) max sup \Mn,2{t,9) - Mn,2itk,e)\ = op{n-^). 

0<fc<«-lo<t-tfc</-i,|e|<7„ 

By (80), (78) and (79), the lemma follows. □ 

Lemma 6. Under conditions of Lemma 5, we have 

(81) sup \Nn{t,e)-Nn{tM=0¥{n^'^ln). 
tG[O,l],|0|<7n 

Proof. Let n„ = supig^^i] |^,|<^,^ \Nn{t,9) - iV„(t,0)|. By (Bl), we have 

JO du 
dv 

Ni{t, u) dudv 

JO 

+ / ' N2{t,u)du+ I ^ N3{t,v)dv, 
Jo Jo 

where u= {u,v)'^ . Hence, 

n„ < / / sup \Ni{t,vL)\dudv + / sup \N2{t,u)\du 
J~'yuJ~'inte[o,i] "'-7nte[0,i] 

(82) 

f 771 



+ / sup \N:>,{t,v)\dv. 
J -In, te[o,i] 

Let Tj = ibn for i = 0, 1, . . . , 6„ and Tj = 1 for i = 6„ + 1, where 6„ = \ h~^\ . 
By the triangle inequality, for u G [— TruTn] x [— TniTn], we have 

sup |A^i(i,u)|< max |iVi(Tj,u)|+ max ^i(u), where 

tG[0,l] 0<j<bn+l l<i<b„+l 

(83) 

Zi{n)= sup |iVi(t,u)-iVi(7i,,u)|. 

By (Bl) and the differentiability of Q(t),Q' (t), Vj(t), Zj^„(t), we conclude 
that -/Vi(t,u) is differentiable with respect to t. Furthermore, simple calcu- 
lations show that 

,n(^,U,t))]wfc,„(i,t,u), 

k=2ieAf„{t) 
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where Rk,n{i,vi,t) = Fk{i/n,Q{t) + {i/n-t)Q' {t) + Tj_i) and Wfc,n(«, 

t,u) satisfies 

(85) sup sup ^ |wfc^„(z,t,u)p = 0(n/6„), A; = 2, 3. 

t6[0,l]|u|<7nie_v-„{t) 

Using (85), (Bl) and similar arguments as those in the proof of Lemma 1, 
we have 



(86) 



sup sup 

te[o,i] |u|<7^ 



dNi{t,\i) 



dt 



By (86), we have for z = 1, 2, . . . , 6„ + 1 



(87) 



l^i(u)||< 





dNi{s,u) 


'n-bn 


dt 



:0((n/6„)V2). 



ds = 0{{nhnf'^). 



max l^i(u) 

i<i<fe„+i 



Since max^^.<^^_^^ l-^i(u)P < Ei=t^ l-^i(u)P, we have 

--0{{nhnf'\l/hnf'^) = 0{n^l^). 

Similarly, || max^^^^^^^-^ |A^i(rj, u)| || = 0(n^/^). By (83), we conclude 

sup |iVi(t,u)| =0{{nbnf'\l/bnf'^) = 0{n^'^). 
te[o,i] 

Same inequality holds with A^i(t,u) herein replaced by N2{t,u) or N3{t,v). 
Thus by (82), ||n„|| = 0((n6„)i/2^„(l/6„)i/2) = 0(n^/2^„) and this lemma 
follows. □ 

Lemma 7. Recall e{t) = [Q{t) - Q{t),hn{Q' [t) - Q'{t))]^ and 7r„ = 
(n6„)~^/^[logn + (6n)~^^^ + (^^n)^^^]- Under conditions of Theorem 3, we 
have suptg[o,i] \0{t)\ = Op(7r„). 

Proof. By (47), it suffices to show that 

(89) sup \m\ = Op((n6„)i/V„) := Op(7f„), 

te[o,i] 

where we recall (45) for '&{t). Define 

n 

A„(t, 7?) = ^[p(yi(t) - i9^Ui,„(t)) - p{Yi{t))]Kk,Xi/n - t), 

i=l 

n 

H„(t, 7?) = An{t, ^) + Y^ ??Tu,,„(t)V'(y.(t))i^6„ {i/n - t) 

i=l 
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(90) 



n 

En{t,^) = Y^mm) - T?^Ui,„(t)) - ^lJ{Yi{t))]ui,nit)KbA^/n - t), 
1=1 

where Uj^„(t) is defined in (46). Let ■jn be a positive sequence such that 
(91) 7n^oo and 7„(n6„)~-^/^ ^ 0. 

Using the same martingale part and differentiable part decomposition tech- 
nique apphed in Lemmas 5 and 6, we can show that 



(92) 

and 
(93) 



sup sup |H„(t,i9) - EH„(t,??)| 

i6[0,l] |i9|<7„ 



:Op(7y'(n6„)-^/^log?i + 7n(n6n)"^/'6;;^/=^) 



sup \^n{t) - mnim = Op(logn + (6„)-^/2). 

t6[0,l] 



On the other hand, simple calculations show that sup^gjQ |E[^'„(t)]| 
Op((n6^)i/2)^ Together with (93) we conclude that 



(94) 



sup |^'„(t)| =Op(^,,). 

te[o,i] 



Furthermore, using the similar arguments as those in the proof of Lemma 2 
and (49), it is easy to see that for 7.„ satisfying (91) 

(95) sup sup |E[H„(^,^?)]-/(^,Q(^))|^9|V2|=o(7^,)• 

^e[o,l] |i?|<7„ 

Note H„(t,i9) = H„(t, s'd) ds. Relation (92) implies that 

sup sup \En{t,l9) -EEn{t,'d)\ <Jn SUp SUp \En{t,'d)-EEnit,l9)\ 

te[o,i] |i?|<7n te[o,i] |i?|<7„ 



which by (95) implies 



(96) 



sup sup 
te[o,i] |t?|<7. 



Ut,^)-^f{t,Q{t)M^ 



^ OF ill) 



For a sequence {c„} with c„ — > oo, let = min{c„, (7r„) ^/^}. Then c'„ — > oo. 
It is easy to see that 7^ = c^vf„ satisfies (91). On the other hand, if |??| = 7„, 
we have by (94) that sup^g^^i] |i9'^*n(t)| = Op(c^7f2). Since inf^gjo,!] f{t, Qa{t)) > 
0, by (96), we have 



(97) 



inf inf AJt,'d)<0 
te[o,i] |i'|=7„ 



0. 
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Note that A„(i, -i?) is convex in and A„(t, 0) = 0. Hence for any ■!? such that 
> %, we have A„(t,??) > {\i^\/%)An{t,^n^M).^ By (97), 
P[infjg[o^i] inf|^l>^^ A„(t, -i?) < 0] — > 0. By the definition of we have 

P[supjg[o,i] ^ c^^n] 0. Since c„ — > oo can be arbitrarily slow, (89) 

follows. □ 

Remark 1. Note that under conditions of Theorem 3, -7r„ — > 0. Hence 
Lemma 7 implies that Q{t) is a uniformly consistent estimator of Q{t) on 
[0,1]. Additionally, if — > cxd, then Q'{t) is also uniformly consistent for 
Q'it) on [0,1]. 

Lemma 8. Under (B2), sup^g^,!] \Sn{t,0{t))\ =Op(l). 

Proof. From the proof of Corollary 2 in Wu (2007a) or Lemma A. 2 in 
Ruppert and Carroll (1980), it is easy to see that 

n 

(98) sup \Sn{t,e{t))\< sup ^|vi(t)|/{X, = Q(t)^z,,„(t)}, 

te[o,i] te[o,i]i=i 

where Q{t) = {Q{t),bnQ'(t)). Note that 

n 

sup J2\^MH^i = QitV^iAt)} 

n 

<C sup ^/{Xi = Q(t)Tz,,„(t)} 
te[o,i] j=i 

n 

= C sup ^ I{Xi = Q{t) - tQ'it) + Q'{t)i/n] 

t6[0,l] j=i 



< C sup ^ /{Xj = ai + a2«/n} , 
where a= (01,02)"''. Following Babu (1989) and using (B2), we see that 



sup ^ I{Xi = oi + a2i/n} = Of {I). 

Therefore this lemma follows. □ 

Proof of Theorem 4. It suffices to prove (20). By (17), we have 



'nhn 



Qa{t) - ^ Wn{t, i)Qa{i/n] 
i=l 



Wn{t,i)Ta^n{i/n) _ ^^^^^ 

~{ f{i/n,Qa{i/n))^/nK, 
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Since sup|^_(|<5^ |rQ,,„(s)| = Op(n6„£»„), YA=iWn{t,i)Qa{i/n) - Qa{t) 
0{hl) and, by (B1HB3), sup|,/„_,|<5„ Qa(i/n)) - Q,(t))| 

0{bn), we have 



"n 



hnf{t,Qa{t))[Qa{t) - Qa{t)] - ^Wn{t, i)Ta,nii /n) / \/nb^ = Op{l) . 

1=1 

(99) 

Basic manipulations similar as those in the proof of Proposition 6 show that 

Ta,nii/n) - Ta^nit) - E{Ta,nii/n) - Ta,n{t)} 

i=l 



o{hn/hn) 



(100) 

Since K £ fC, it is easily seen that sup|jy„_^|<5^ |E{Ta^„(i/n) — Ta^n{'t)}\ 
o{y/rib^). Combining (99) and (100), we have (20). □ 



Outline of the Proof of Theorem 5. From the proofs of Lemmas 
5 and 6, we can obtain supgg_y\/-^(j) \Qa{s) — Qa{s)\ = Op(£'n)- Using the mar- 
tingale decomposition technique therein, it can be shown that, if we replace 
the estimates Q{i/n) and Q{t) in (25) and (26) by the true values Q{i/n) 
and Q(t), the changes of (25) and (26) will be of order op(l). By the property 
of local stationarity, the same conclusion holds when we further replace Zi ^^ 
and Xi in (25) and (26) by Cj = V'a(-^j — Q{^)) and Xi = G{t,Ti), respec- 
tively. Applying convergence results for the stationary processes {ci} and 
{Xi}, Theorem 5 follows. A detailed proof is available upon request. □ 

Acknowledgments. We are grateful to three referees, the associate editor 
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